Skip to content

Drop four fuzzer-self-noise sources found across three PyPy fleets - #265

Merged
devdanzin merged 1 commit into
mainfrom
fleet-noise-self-echo-and-sigint
Aug 27, 2026
Merged

Drop four fuzzer-self-noise sources found across three PyPy fleets#265
devdanzin merged 1 commit into
mainfrom
fleet-noise-self-echo-and-sigint

Conversation

@devdanzin

Copy link
Copy Markdown
Owner

Every one of these is fusil scoring text the target echoed rather than a crash the target reported. Together they accounted for 8 kept crash dirs across PyPy fleets 05–07.

1. help() prints fusil's own comments

pydoc.getdoc falls back to inspect.getcomments() for an object with no docstring, so the comment block above an undocumented class in a spliced sample file is printed verbatim by help(obj) — and _sitebuiltins._Helper.__call__ is a normal fuzz target.

Match pattern b'segfault' in b' |  # path overflows its C/Rust stack -> segfault. ...'

Two comments in samples/tricky_objects.py carried the word. Reworded, plus SelfNoiseVocabularyTests scans the emitted script's comment lines for the 1.0 vocabulary so it cannot creep back. Only comments are checked — the same words legitimately occur in real emitted code (SystemError in the bomb list, AssertionError in a class statement), which inspect.getcomments never reaches.

2. A traceback is the target quoting itself — in two shapes

shape example fleet
source line LOGGER.critical('Future %s in unexpected state: %s', 05
frame line File ".../logging/__init__.py", line 1536, in critical 06

concurrent.futures._base kept a dir via each half. The frame-line rule requires the comma Python tracebacks put before in; faulthandler writes line N in func without one, so genuine fatal-signal reports still score — pinned by a test.

3. bdb's tracer echoes traced values

Match pattern b'systemerror' in b"+++ return <class 'SystemError'>"

The repr of an arbitrary fuzz value routinely holds a scored word, so tracing any module under bdb/pdb manufactures crashes. Ignores the tracer prefixes (+++, ---, !!!); no diagnostic uses them.

4. SIGINT handlers called directly as fuzz targets

asyncio's Runner._on_sigint and pdb.sigint_handler both raise KeyboardInterrupt() unconditionally. That BaseException escapes the generated script's except Exception handlers, and an uncaught KeyboardInterrupt makes the interpreter re-raise SIGINT — so the process looks killed by signal 2 and WatchProcess scores it 1.0.

_on_sigint was already blacklisted by #263 as "asyncio.runners:Runner". But Runner is re-exported as asyncio.Runner, so a session whose target module is asyncio never matches that key — and reached it anyway through the runtime generic-method loop. pdb.sigint_handler was never keyed at all. Both now live in METHOD_BLACKLIST, which is name-based and module-agnostic, the same mechanism that already covers raise_signal. The module-keyed entry stays: it still filters the static generation path for the defining module.

WatchProcess is deliberately left alone — fixing the cause beats masking the symptom, and changing how signals are scored is far riskier than a denylist entry.

Note on the golden snapshot

It moves because METHOD_BLACKLIST is emitted into every generated script as _FUSIL_METHOD_BLACKLIST — which is exactly why the name-based fix reaches the paths the module-keyed one missed.

Verification

1277 tests pass; ruff check and ruff format --check both clean. Each fix was verified to fail without the change (the vocabulary test was checked by reintroducing the word).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WPBSmN87d2BqUnrDbojUT1

Every one of these is fusil scoring text the TARGET echoed rather than a crash
the target reported. Together they accounted for 8 kept crash dirs across
fleets 05-07.

1. help() prints fusil's own comments.  pydoc.getdoc falls back to
   inspect.getcomments() for an object with no docstring, so the comment block
   above an undocumented class in a spliced sample file is printed verbatim by
   help(obj) -- and _sitebuiltins._Helper.__call__ is a normal fuzz target. Two
   comments in samples/tricky_objects.py contained "segfault" (a 1.0 word) and
   scored a session 100%. Reworded, plus SelfNoiseVocabularyTests scans the
   emitted script's COMMENT lines for the 1.0 vocabulary so it cannot creep
   back. Only comments are checked: the same words legitimately occur in real
   emitted code (SystemError in the bomb list, AssertionError in a class
   statement), which inspect.getcomments never reaches.

2. A traceback is the target quoting itself, in two shapes. The source line
   (`LOGGER.critical('Future %s in unexpected state: %s',`) and the frame line
   (`File ".../logging/__init__.py", line 1536, in critical`), where the scored
   word is the crashing function's NAME. concurrent.futures._base kept a dir via
   each half in two different fleets. The frame-line rule requires the comma
   Python tracebacks put before `in`; faulthandler writes `line N in func`
   WITHOUT one, so genuine fatal-signal reports still score -- pinned by a test.

3. bdb's tracer echoes every traced event and the value involved
   (`+++ return <class 'SystemError'>`). The repr of an arbitrary fuzz value
   routinely holds a scored word, so tracing any module under bdb/pdb
   manufactures crashes. Ignore the tracer prefixes; no diagnostic uses them.

4. SIGINT handlers called directly as fuzz targets. asyncio's Runner._on_sigint
   and pdb.sigint_handler both `raise KeyboardInterrupt()` unconditionally; that
   BaseException escapes the generated script's `except Exception` handlers, and
   an uncaught KeyboardInterrupt makes the interpreter re-raise SIGINT, so the
   process looks "killed by signal 2" and WatchProcess scores it 1.0.

   _on_sigint was already blacklisted as "asyncio.runners:Runner" (#263), but
   Runner is re-exported as asyncio.Runner, so a session whose target module is
   asyncio never matches that key -- and reached it anyway through the runtime
   generic-method loop. pdb.sigint_handler was never keyed at all. Both now live
   in METHOD_BLACKLIST, which is name-based and module-agnostic, the same
   mechanism that already covers raise_signal. The module-keyed entry stays: it
   still filters the static generation path for the defining module.

   WatchProcess is deliberately left alone. Fixing the cause beats masking the
   symptom, and changing how signals are scored is far riskier than a denylist
   entry.

The golden snapshot moves because METHOD_BLACKLIST is emitted into every
generated script as _FUSIL_METHOD_BLACKLIST -- which is exactly why the
name-based fix reaches the paths the module-keyed one missed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WPBSmN87d2BqUnrDbojUT1
@devdanzin
devdanzin merged commit 811111e into main Aug 27, 2026
0 of 2 checks passed
@devdanzin
devdanzin deleted the fleet-noise-self-echo-and-sigint branch August 27, 2026 08:44
Nortaq-PlayNexus pushed a commit to Nortaq-PlayNexus/fusil that referenced this pull request Sep 4, 2026
… level constant

Both measured on one PyPy --concurrency-stress fleet (158 827 sessions, 229 kept
dirs). Together they account for 177 of those 229 dirs -- 77% of everything the
fleet kept was fusil hurting itself or scoring an echo.

socket.close/dup/fromfd/send_fds take a RAW INTEGER file descriptor, so any fuzz
integer closes or reinterprets a descriptor the interpreter is still using. This is
the int-as-FD analogue of the int-as-pointer family already excluded in CTYPES, and
it is not a target defect: CPython aborts on it too, measured 1/6 against PyPy's 6/6
(the gap is window width, not correctness).

163 of 229 dirs, in two faces that split exactly on the value passed. 120 closed
some other descriptor and were captured with glibc's own `Unexpected error 9 on
netlink descriptor 11` -- 11 being socket.AF_ROSE, which the stress region had
picked as a shared object. The other 43 were SIGABRTs with an EMPTY stdout, and all
43 of them shared a constant worth 0, 1 or 2: they had closed the child's own stdout
or stderr, so the diagnostic had nowhere to go. That is why the bucket looked like
two problems.

The entry is module-keyed (BLACKLIST["socket"]), not name-based: `close` and `dup`
are also methods on socket, file and many other objects, where they take no
descriptor and are fine to fuzz. A METHOD_BLACKLIST entry would stop fusil ever
closing anything.

The second fix is a third face of the self-echo disease PR devdanzin#265 addressed, and one
that survived it. `warnings` prints the source line of whatever frame it fired in:

  /.../logging/__init__.py:1536: RuntimeWarning: coroutine '...' was never awaited
    self._log(CRITICAL, msg, args, **kwargs)

so any un-awaited coroutine collected while logging is on the stack echoes logging's
own source, and `CRITICAL` there is the level constant passed as an argument. It is
neither a `File "...", line N, in ...` frame nor a `.critical(` call, so neither devdanzin#265
rule reaches it: 14 kept dirs across asyncio.base_events, asyncio.streams and
asyncio.selector_events. The new rule matches the constant only in an argument
position and is case-sensitive, so a real record (`CRITICAL:root:...`) and English
prose both still score.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant